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DETAILED ACTION 

1 . This office action is responsive to application 1 0/657,21 0 filed September 9 th , 
2003. Claims 1-19 have been examined. 

2. Receipt is acknowledged of papers submitted March 8 th , 2004: Response to pre- 
exam formalities, including editing figures' numbers/letters/reference characters to meet 
size requirements pursuant to 37 C.F.R. 1.84 and 37 C.F.R. 1.121. 

Specification 

Applicant is reminded of the proper language and format for an abstract of the 
disclosure: 

The abstract should be in narrative form and generally limited to a single 
paragraph on a separate sheet within the range of 50 to 150 words. It is important that 
the abstract not exceed 150 words in length since the space provided for the abstract 
on the computer tape used by the printer is limited. The form and legal phraseology 
often used in patent claims, such as "means" and "said," should be avoided. The 
abstract should describe the disclosure sufficiently to assist readers in deciding whether 
there is a need for consulting the full patent text for details. 

The language should be clear and concise and should not repeat information 
given in the title. It should avoid using phrases which can be implied, such as, "The 
disclosure concerns," "The disclosure defined by his invention," "The disclosure 
describes," etc. 

3. The abstract of the disclosure is objected to because improper claim language. 
Correction is required. See MPEP § 608.01(b). 
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Claim Objections 

4. Claims 5-6 are objected to because of the following informalities: These claims 
state "...from said neural network parts," where the claims from which 5 and 6 depend 
only mention a singular neural network part. The examiner has assumed for the rest of 
the prosecution of this application that these claims meant to recite a singular neural 
network part, however, appropriate correction is required. 

Regarding claim 16 : 

5. Claim 16 is objected to as being dependent upon a rejected base claim, but 
would be allowable if rewritten in independent form including all of the limitations of the 
base claim and any intervening claims. 

Claim Rejections - 35 USC § 103 

The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

6. Claims 1-8 t 10-15, and 17-19 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Manevitz et al. (Finite-Element Mesh Generation Using Self- 
Organizing Neural Networks, Computer-Aided Civil & Infrastructure Engineering, July 
1997, Vol. 12 Issue 4, page 233; herein referred to as Manevitz) in view of Chedid et al. 
(IEEE Transactions on Magnetics: Vol. 32, No. 5, September 1996; herein referred to 
Chedid). 
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Regarding claim 1: 

Manevitz teaches an apparatus for calculating numerical solutions for partial 
differential equations in successive intervals using adaptive meshes and where said 
mesh adaptively refines itself about emerging regions of complexity as said partial 
differential equation progresses over said successive intervals: 

Neural networks are applied to the problem of mesh placement 
for the finite-element method. When the finite element method 
is used to numerically solve a partial differential equation with 
boundary conditions over a domain, the domain must be 
divided into "elements." (abstract, lines 1-5) (Manevitz) 

The mesh should be finer in regions where the solution is 
believed to be changing rapidly or to have large gradients. 
Thus smaller elements should be used near singularity points 
such as reentrant corners or cracks, near holes, near small 
features of the boundary, near the location of rapidly changing 
boundary data, at and near inhomogeneities, etc. (§1, H6, 
subsection 1) (Manevitz) 



Manevitz fails to teach using the neural net to produce predictions of values of a 
parameter following intervals based on values of said parameter available from previous 
intervals. 

Chedid teaches a neural network part for producing predictions of values of a 
parameter at a following interval based on values of said parameter available from 
previous intervals. 

a prediction of mesh density is accomplished by a simple 
feedforward neural network which has the ability to learn the 
relationship between mesh density and model geometric 
features. It will be shown that ANN are able to recognize 
delicate areas (abstract, lines 14-18) (Chedid) 

a technique to predict the mesh density distribution, based on 
artificial neural networks. The basis is the fact that these 
networks have the ability to learn from examples and to 
generalize. It will be shown that a simple feedforward ANN is a 
suitable candidate, and that reduction in both computation 
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time and complexity can be achieved through the introduction 
of the layer-by-layer approach, which will be described later in 
Section IV-C. The predicted mesh density function can then be 
used in conjunction with a density-driven software [3] , [5] or, as 
will be shown in Part II of this paper, sent to a Kohonen 
network to predict the final mesh (§ 1, fl 3, lines 22-33) 
(Chedid) 

It would have been obvious for one of ordinary skill in the art at the time of 
applicant's invention to have combined Manevitz's neural network approach to a finite 
element method (FEM) application of solving partial differential equations (PDEs) with 
Chedid's predictive feedforward neural network (also specifically a Kohonen network) to 
considerably hasten computation of the next adaptation of the mesh as applied to the 
domain of the solution to the PDE. Applicant has admitted the community awareness of 
such a system. Both neural networks being used for prediction of values of a parameter 
at a following interval based on values of said parameter available from previous 
intervals, and adaptive meshing through FEM of PDEs are known to have existed prior 
to applicant's invention. Therefore, combining the predictive behavior of such ANNs to 
look ahead in the FEM thereby reducing computation time would have been obvious to 
one of ordinary skill in the art. 

Regarding claim 2: 

Manevitz discloses the limitations of claim 2 wherein said successive intervals 
are time intervals. The successive intervals are the discretized "rapidly changing data" 
(§1, subsection 1) (Manevitz) about which the regions corresponding to the mesh 
Manevitz discloses should be finer to capture the more interesting and detailed data. 
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Regarding claim 3: 

Manevitz discloses the limitations of claim 3 wherein said parameter is a 
gradient. 

The mesh should be finer in regions where the solution is 
believed to be changing rapidly or to have large gradients. (§ 1, 
subsection 1) (Manevitz) 

Regarding claim 4: 

Manevitz discloses the limitations of claim 4 wherein said mesh adaptation part is 
further configured to adaptively coarsen said mesh about regions of low complexity: 

Manevitz mentions a need to have the mesh finer in regions of high complexity 
and that it should not be uniform, and also mentions the high computational cost of 
rendering such detailed information (§1, 1J5), wherein said mesh adaptation part is 
further configured to adaptively coarsen said mesh about regions of low complexity. 

Of course, the density of the mesh should not necessarily be 
uniform. The mesh may be finer in some regions and coarser in 
others ... Thus the actual density of the mesh used in a certain 
computation is a compromise between accuracy and cost. ... 
The mesh may be finer in some regions and coarser in others. 
(§1,H5) (Manevitz) 

The common trade-off of computation cost and resource expenditure for 
accuracy and precision has been common in computer science since its beginnings as 
well as elsewhere. It would have been obvious for one of ordinary skill in the art at the 
time of applicant's invention to employ techniques to minimize the computation cost and 
make efficient use of limited resources, such techniques including but not limited to 
coarsening the mesh density and thus somewhat alleviating intensive data processing. 
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Regarding claims 5 and 6: 

Manevitz discloses the limitations of claims 5 and 6, wherein said mesh 
adaptation part comprises a first/second thresholder for thresholding gradients from said 
neural network partes), such that gradients above/below said threshold value are taken 
to indicate complexity and to lead to local refining/coarsening of said mesh 
(respectively) (see above quote from Manevitz, §1, 5). 

Examiner holds thresholding to be inherent in automatically and quantitatively 
classifying elements to be either refined or coarsened. Application of the gradient as 
the signaling characteristic would be an obvious choice of stated thresholders to one of 
ordinary skill in the art, as a gradient is a measure of the rate of change of some 
quantitative property, which is ideal for the purpose of determining regions of 
complexity. 

Regarding claim 7: 

Manevitz and Chedid teach the limitations of claim 1 for the reasons above, and 
in addition Chedid discloses the limitations of claim 7 wherein said neural network part 
comprises two neural networks, each having an input layer of input elements, at least 
one hidden layer of hidden elements and an output layer of at least one output element, 
said two neural networks differing from each other in respective numbers of input 
elements. 
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Examiner holds this configuration of neural networks wherein having an input 
layer of input elements, at least one hidden layer of hidden elements and an output 
layer of at least one output element, to be standard of innumerable types of neural 
networks containing a hidden layer (feedforward, recurrent, radial-basis-function, etc) 
and is by no means novel to one of ordinary skill in the art at the time of applicant's 
invention. 

The network consists of an input layer, one or more hidden 
layers, and an output layer [8]. Signals flow into the input layer, 
pass through the hidden layer, and arrive at the output layer. (§ 
IV, subsection B, lines 1-4) (Chedid) 

This configuration wherein 2 neural networks differ from each other in respective 
numbers of input elements is an example of robustness for similar tasks, as well as 
parallel computation for potentially either similar or different tasks. 

Regarding claim 8: 

Manevitz and Chedid teach the limitations of claim 1 for the reasons above, and 
in addition Chedid discloses the limitations of claim 8, wherein each hidden element 
defines a hyperbolic tan-sigmoid transfer function. 

Manevitz fails to teach the apparatus of claim 7 wherein each hidden element 
defines a hyperbolic tan-sigmoid transfer function. This is a common and flexible 
transfer function with which to bound the hidden layer node's input. 

Chedid discloses a neural network wherein each hidden element defines a 
hyperbolic tan-sigmoid transfer function: (a hyperbolic tan-sigmoid transfer function 
function is a sigmoid function) 
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The network consists of an input layer, one or more hidden 
layers, and an output layer [8]. Signals flow into the input layer, 
pass through the hidden layer, and arrive at the output layer. 
With the exception of the input layer, each neuron receives 
signals from the neurons of the previous layer linearly weighted 
by the interconnect values between neurons. The neuron then 
produces its output signal by passing the summed signal through 
a sigmoid function. (§ IV, subsection B, lines 1-9) (Chedid) 



Regarding claim 10: 

Manevitz discloses the limitations of claim 10, wherein a first of said neural 
networks is a boundary element neural network for calculating gradients of boundary 
elements of said adaptive mesh, and a second of said neural networks is an interior 
element neural network for calculating gradients of interior elements of said adaptive 
mesh. 

One novelty of the method is the interweaving of versions of the 
Kohonen algorithm in different dimensions simultaneously in 
order to handle the boundary of the domain properly, 
(abstract, 2 nd H) (Manevitz) 

For our application, it is important that eventually the network , 
i.e., that boundary points of the network fall on boundaries 
of the domain. The basic Kohonen map does not typically fulfill 
this constraint. For example, for the body in Fig. 5, the result is 
as in Fig. 6. This constraint resulted in the implementation of 
some heuristic rules and the interweaving of several Kohonen 
algorithms together... (§ 3.2.3) (Manevitz) 

In order to distribute the points appropriately on the 
boundary (but after the net has reached the boundary), we 
apply a Kohonen algorithm to this one-dimensional simplex. 
(§3.2.4, H 3) (Manevitz) 

In addition, once a point is on the boundary, we do not allow it 
to leave the boundary (as might occur from the two dimensional 
algorithm); in effect, the two-dimensional algorithm now acts 
only on the interior of the original net. (§ 2 - § 3) (Manevitz) 
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The claimed first boundary element neural network for calculating gradients of 
boundary elements of said adaptive mesh, and a second of said neural networks is an 
interior element neural network for calculating gradients of interior elements of said 
adaptive mesh are the disclosed "self-organizing algorithm of Kohonen [which] is 
adapted to solve the problem of automatically assigning (in a near-optimal way) 
coordinates from a two-dimensional domain to a given topologic grid (or mesh) of nodes 
in order to apply the finite element method effectively when solving a partial differential 
equation with boundary conditions over that domain." (Manevitz, abstract, 

Regarding claim 11: 

Manevitz discloses the limitations of claim 1 1 , wherein said boundary element 
neural network has fewer input elements than said interior element neural network. 

In other words, the neural network becomes a representative 
map of the sample data information. This is exploited by us, in 
order to arrange for the placement of the finite-element mesh, by 
identifying the mesh nodes with neural nodes and identifying the 
weight space with the physical space of the domain, thereby 
causing the network to be an approximation of the density 
function. (§ 2, H 7) (Manevitz) 

Therefore, the amount of input elements is proportional to the effort expended in 
applying the mesh to either the boundary or the interior. Since there are many more 
interior nodes than boundary nodes, the boundary element neural network necessitates 
having fewer input elements than the interior element neural network. 



Regarding claim 12 : 
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Manevitz discloses the limitations of claim 12, wherein said input elements are 
connected to gather for a given mesh element a gradient of said mesh element, and a 
gradient of each neighboring element for each of a current and a previous interval. 

The location of the chosen node is adjusted, as is the 
coordinates of all nodes within a certain neighborhood of this 
chosen node. (§ 3.1, subsection 4) (ALSO Figure 1) 
(Manevitz) 

The mesh should be finer in regions where the solution is 
believed to be changing rapidly or to have large gradients. 

Thus smaller elements should be used near singularity points 
such as reentrant corners or cracks, near holes, near small 
features of the boundary, near the location of rapidly changing 
boundary data, at and near inhomogeneities, etc. (§ 1, H 6, 
subsection 1) (Manevitz) 

Applicant is reminded of the herein discussed intervals in the rejection of claim 1. 



Regarding claim 13 : 

Manevitz discloses the limitations of claim 13, wherein each hidden element 
defines a hyperbolic tan-sigmoid transfer function, each output element defines a linear 
transfer function, a first of said neural networks is a boundary element neural network 
for calculating gradients of boundary elements of said adaptive mesh, and a second of 
said neural networks is an interior element neural network for calculating gradients of 
interior elements of said adaptive mesh, and said boundary element neural network has 
fewer input elements than said interior element neural network. 

Claim 13 is rejected since its limitations are the sum of dependent claims 8 
through 11, all of which have been separately and individually rejected for reasons 
mentioned above. The rejections when taken together stand as a similar rejection for 
claim 13. Please see aforementioned rejections. 
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Regarding claims 14 and 15 : 

Manevitz discloses the limitations of claims 14 and 15 wherein said interior 
element neural network comprises eight input elements, six hidden elements and one 
output element and wherein said boundary element neural network comprises six input 
elements, six hidden elements and one output element. 

The network consists of an input layer, one or more hidden 
layers, and an output layer [8]. Signals flow into the input layer, 
pass through the hidden layer, and arrive at the output layer. 
With the exception of the input layer, each neuron receives 
signals from the neurons of the previous layer linearly weighted 
by the interconnect values between neurons. The neuron then 
produces its output signal by passing the summed signal 
through a sigmoid function [8]. In this work, the network (see 
Fig. 5) consists of an input layer of eight processing units, one 
hidden layer of 12 processing units, and an output layer of 
one processing unit. (§ IV, subsection B: Architecture) 
(Chedid) 



Regarding claim 18 : 

Manevitz discloses a method of adapting a finite element mesh (abstract, 11, 
lines 1-2, 5) interactively with calculations of numerical solutions for a partial differential 
equation (abstract, 11, lines 2-5) in successive intervals (§1, 16, subsection 1), said 
partial differential equation being calculated over a domain in accordance with 
respective finite elements of said mesh (abstract, 11, lines 2-5), each successive 
interval using a further adaptation of said mesh (abstract, 13, lines 3-5), the method 
comprising: 
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producing predictions of values of a parameter of said partial differential equation 
at mesh elements (§1, H6, subsection 1) at a following interval based on values of said 
parameter available from previous intervals, 

and adapting said mesh over said domain of a respective partial differential 
equation using said predictions, such that said mesh adaptively refines itself about 
emerging regions of complexity (abstract, 1J3, lines 3-5), as said partial differential 
equation progresses over said successive intervals. 

Manevitz fails to teach using the neural net to produce predictions of values of a 
parameter following intervals based on values of said parameter available from previous 
intervals or said neural network adapting said mesh over a domain of a respective 
partial differential equation using said predictions. 

Chedid teaches a neural network that produces predictions of values of a 
parameter at a following interval based on values of said parameter available from 
previous intervals (abstract, H 1, lines 14-18, neural network prediction requires 
previous data, or training data, for successful generalization) and having the neural 
network adapt said mesh over a domain using said predictions. 

Regarding claim 19 : 

Manevitz discloses the limitations of claim 19 wherein said parameter is a 
gradient. 

For example, it is expected that the solution exhibits high 
gradients near a sharp corner in the boundary. In such 
cases, a density function may be chosen to exploit this 
information. If nothing is known a priori about the solution, a 
density function may still be chosen in an adaptive a posteriori 
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fashion. For example, the problem may be solved preliminarily 
with a uniform-density mesh (perhaps with relatively few nodes), 
and then a density function may be constructed based on 
the gradients of this solution prior to resolving it. (In our 
examples we chose the density function by hand, with the 
knowledge of the available exact solutions.) (§ 2, H 5, 13-24) 
(Manevitz) 



Claim Rejections - 35 USC § 103 

7. Claim 9 is rejected under 35 U.S.C. 103(a) as being unpatentable over Manevitz 
and Chedid as applied to claims 1 and 7 above, and in further view of Hassoun 
(Fundamentals of Artificial Neural Networks, MIT Press, 1995). 

Manevitz and Chedid fail to teach the apparatus according to claim 9, wherein 
each output element defines a linear transfer function. 

Hassoun teaches a single-hidden-layer net with both sigmoidal activation units 
(of claim 8) and wherein each output element defines a linear transfer function: 

The most important feature that distinguishes the RBF network 
from earlier radial basis function-based models is its adaptive 
nature, which generally allows it to utilize a relatively smaller 
number of locally tuned units (RBFs). ... The RBF network has a 
feedforward structure consisting of a single hidden layer of J 
locally tuned units which are fully interconnected to an output 
layer of L linear units, as shown in Figure 6.1.1. (§6.1, page 
286) (Hassoun) 

It would have been obvious for one of ordinary skill in the art at the time of 
applicant's invention to have combined the linear output transfer function of Hassoun 
with Manevitz and Chedid's automatic finite element mesh generation procedures to 
utilize the benefits of radial-basis-function neural networks, including but not limited to 



accelerated training (when compared to backpropagation neural networks). 
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Claim Rejections - 35 USC § 103 

8. Claim 17 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
Manevitz as applied to claim 16 above, and further in view of Lagaris et al. (Artificial 
Neural Networks for Solving Ordinary and Partial Differential Equations, IEEE 
Transactions On Neural Networks, Vol. 9, No. 5, September 1998; herein referred to as 
Legaris). 

Manevitz and Chedid fail to teach the apparatus according to claim 17, wherein 
said neural network part is configured for training using the Levenberg Marquardt 
training method. 

Legaris discloses a neural network wherein said neural network part is 
configured for training using the Levenberg Marquardt training method. 

Once the derivative of the error with respect to the network 
parameters has been defined it is then straightforward to employ 
almost any minimization technique. For example it is possible 
to use either the steepest descent (i.e., the backpropagation 
algorithm or any of its variants), or the conjugate gradient 
method or other techniques proposed in the literature. In our 
experiments we have employed the quasi-Newton BFGS method 
[9] (§ II, subsection A: Gradient Computation, H5) (Legaris) 

It would have been obvious for one of ordinary skill in the art at the time of 
applicant's invention to apply the Levenberg Marquardt training method to improve the 
speed of convergence to local minima while also escaping shallow local minima, which 
generally leads to better solution quality. (See Hassoun, page 218) 
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Conclusion / Correspondence Information 

9. Claims 1-15, 17-19 are rejected. 

10. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure: 

• Jerome T. Connor, R. Douglas Martin, L. E. Atlas; Recurrent Neural Networks 
and Robust Time Series Prediction; IEEE TRANSACTIONS ON NEURAL 
NETWORKS, VOL. 5, NO. 2, MARCH 1994 

• Mike L. Forcada; Neural Networks: Automata and Formal Models of 
Computation; January 21, 2002; http://www.dlsi.ua.es/~mlf/nnafmc/ (available 
June 5 th 2002) 

• D. A. Lowther; On Automatic Mesh Generation using Kohonen Maps; IEEE 
TRANSACTIONS ON MAGNETICS, VOL 34, NO. 5, SEPTEMBER 1998 

• D. G. Triantafyllidis & D. P. Labridis; An Automatic Mesh Generator For Handling 
Small Features In Open Boundary Power Transmission Line Problems Using 
Artificial Neural Networks; Commun. Numer. Meth. Engng. 2000; 16:177-190 

• D. Gobovic and M.E. Zaghloul; Analog Cellular Neural Network With Application 
to Partial Differential Equations With Variable Mesh-Size 



1 1 . Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Michael Caldwell whose telephone number is (571) 272- 
1942. The examiner can normally be reached on Mon-Fri 10:00-6:00. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, David Vincent can be reached on (571) 272-3080. The fax phone number 
for the organization where this application or proceeding is assigned is 571-273-8300. 
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Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-21 7-91 97 (toll-free). 
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